Over the past decade, the Internet, or more specifically, the Internet Protocol (IP) based network, has seen a tremendous growth. The proliferation of the Internet and the increasing number of Internet users has resulted in extension and scaling problems for applications. This is especially true for applications designed for end-users, such as the World Wide Web (WWW), and audio-visual streaming. The increased in network bandwidth and processing power can hardly catch up with the demands of the increasing number of Internet users. This has resulted in longer load time for WWW page request, and lost of quality in real-time audio-visual playback across the Internet. The effort to reduce such undesirable effects has led to a wide deployment of intelligent network elements at the network edge (i.e. nearer to the end-users).
The most common use of such intermediate network elements are to function as caching proxies, such as hyper text transfer protocol (HTTP) proxies and/or caches as described in an article “Hypertext Transfer Protocol—HTTP/1.1”, IETF RFC 2616, June 1999, by Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee. These have been successful in reducing the network load at the WWW server and accelerating WWW contents delivery to the end-users. However, as the number of end-users increases, the variety of web-browser configurations and platforms widens. Similarly, the range of web contents is also broadening. Simply replicating static web contents cannot hope to sustain the ever-increasing demands from the end-users.
In addition, there has also been a noted increase in the deployment of multimedia streaming over the Internet. These usually employ the real-time streaming protocol (RTSP) as session protocol to set up and tear down communications channel, and real-time transport protocol and real-time control protocol (RTP/RTCP) for the actual transmission of content data. The RTSP is disclosed for example, in “Real Time Streaming Protocol (RTSP)”, IETF RFC 2326, April 1998, by Schulzrinne, H., Rao, A., and Lanphier, R. The RTP/RTCP is disclosed for example, in “RTP: A Transport Protocol for Real-Time Applications”, IETF RFC 1889, January 1996, by Schulzrinne, H., Casner, S., Frederick, R., and Jabcobson, V. Because of the versatile nature of Internet traffic, adaptation of the multimedia stream to the fluctuating traffic conditions is necessary to ensure a smooth presentation to the end-user. Though RTCP provides a means for end-users to report their communication status back to the sender, measures taken up by the sender based on receiver report can hardly be effective because the distance (network-sense) between the receiver and sender is often large. As most end-users connects to the Internet via an intermediary of some sort, for instance, firewall gateways, Network Address Translator (NAT), or proxies, the intermediary present a good choice to perform adaptation services on behalf of the content originator.
Furthermore, as the Internet grows, so does the range of devices that are used to access content from the Web. This diversification of browser types has accelerated with recent advancements in wireless Internet technology, in which tiny handheld devices such as digital personal assistants (PDA) and mobile phones have built-in micro-browsers that browse the Web, or play back audio/visual streams. Content authors can no longer develop content with the assumption that the created content will only be viewed by users using traditional desktop computers. Device independence is now a critical consideration, as disclosed in “Device Independence Principles”, W3C Working Draft, September 2001, by Gimson, R., et al.
A number of international standardization organizations have recognized the need to provide services originally available only at the network core (where the servers are located) to the network edge (where the end users are located). For instance, the Internet Engineering Task Force (IETF) has recently set up working groups focusing on providing services at the network edges. The Open Pluggable Extensible Services (OPES) working group is one such effort. The OPES working group focuses on extending the current HTTP proxies from performing simple caching tasks to a whole suite of adaptation services. The framework of OPES is specified in “A Model for Open Pluggable Edge Services”, IETF Intemet Draft, Work In Progress, November 2001, by Tomlison, G., Chen, R., and Hofmann, M. There is also a Content Distribution Internetworking (CDI) working group that concentrates on collaborations between different content distribution networks (CDN). Such collaboration efforts are believed to be able to further accelerate the delivery of contents to the end user.
The current use of intermediaries in content delivery is mostly restricted to providing simple functionality such as HTTP caching, HTTP proxy, or RTSP proxy. This cannot hope to maintain the service level demanded by the users of today's Internet, as the number of end-users increases exponentially. Moreover, with the range of hardware devices and software agents employed to retrieve contents by different users are also broadening, content providers are finding it difficult to present to the users a coherent set of contents are that suited to the user's device and preferences.
Though various international bodies have recognized the above problems, and have acted to provide resolutions, their work could still be improved on. The OPES framework described in focused on the operations of a single intermediary, ignoring the current trend of collaborations between content delivery networks. In addition, though the idea of the OPES framework is to perform content adaptation so as to enhance the user experience in content retrieval, it focused only on parameters of the HTTP. This is not only inadequate for device independence, it also does not cater to the growing number of audiovisual streaming applications.